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Abstract 


We  have  developed  methods  for  finding  globally  optimal  solutions  to  various  classes 
of  nonconvex  optimization  problems.  We  have  shown  that  any  nonconvex  conic  quadrat- 
ically  constrained  quadratic  program  can  be  lifted  to  a  convex  conic  optimization  prob¬ 
lem.  We  have  shown  that  a  complementarity  approach  can  be  used  to  find  sparse 
solutions  to  optimization  problems,  with  promising  initial  theoretical  and  computa¬ 
tional  results.  We  have  investigated  various  relaxation  approaches  to  several  classes 
of  problems  with  complementarity  constraints,  including  linear  programs  with  com¬ 
plementarity  constraints,  support  vector  regression  parameter  selection,  bi-parametric 
linear  complementarity  constrained  linear  programs,  quadratic  programs  with  comple¬ 
mentarity  constraints,  and  nonconvex  quadratically  constrained  quadratic  programs, 
proving  various  theoretical  results  for  each  of  these  problems  as  well  as  demonstrating 
the  computational  effectiveness  of  our  approaches. 


1  Introduction 

Six  papers  have  been  published  (see  the  archival  publications  section).  Four  additional 
papers  have  been  submitted: 

[A]  “On  QPCCs,  QCQPs  and  Completely  Positive  Programs”,  by  L.  Bai,  J.E.  Mitchell, 
and  J.-S.  Pang.  Journal  submission. 

[B]  “Complementarity  Formulations  of  fVnorrn  Optimization  Problems”,  by  M.  Feng, 
J.E.  Mitchell,  J.-S.  Pang,  X.  Shen,  and  A.  Wchter.  September  23,  2013.  Journal  submission. 

[C]  “An  Algorithm  for  Global  Solution  to  Bi-Parametric  Linear  Complementarity  Con¬ 
strained  Linear  Programs”,  by  Y.-C.  Lee,  J.-S.  Pang,  and  J.E.  Mitchell.  Journal  submission. 

[D]  “Global  Resolution  of  the  Support  Vector  Machine  Regression  Parameters  Selection 
Problem”,  by  Y.-C.  Lee,  J.-S.  Pang,  and  J.E.  Mitchell.  Journal  submission. 

Most  of  these  papers  are  available  from  Mitchell’s  webpage, 


http : //www . rpi . edu/~raitchj 

Four  doctoral  students  who  were  partially  supported  by  this  grant  have  graduated  from 
RPI  or  UIUC: 

•  Lijie  Bai,  On  convex  quadratic  programs  with  complementarity  constraints,  August 
2013,  RPI. 

•  Tim  Lee,  Approximations  and  improvements  to  semidehnite  relaxations  of  optimiza¬ 
tion  problems,  August  2013,  RPI. 

•  Yu-Ching  Lee,  Global  solution  to  parametric  complementarity  constrained  programs 
and  applications  in  optimal  parameter  selection,  August  2013,  UIUC. 

•  Bin  Yu,  A  Branch  and  Cut  Approach  to  Linear  Programs  with  Linear  Complemen¬ 
tarity  Constraints,  August  2011,  RPI. 
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Further  papers  are  in  preparation,  including  documentation  of  methods  for  finding  global 
optima  of  linear  programs  with  complementarity  constraints.  Multiple  talks  have  been  given 
at  conferences  and  universities. 

In  paper  [A] ,  we  show  that  any  quadratically  constrained  quadratic  program  is  equivalent 
to  a  convex  optimization  problem.  The  result  requires  no  assumptions  on  the  boundedness 
of  the  feasible  region  or  on  the  convexity  of  the  quadratic  constraints.  The  result  also  holds 
if  a  nonnegativity  assumption  on  the  variables  is  replaced  by  a  more  general  convex  conic 
constraint;  for  example,  the  result  holds  if  the  variables  satisfy  a  second  order  cone  constraint 
or  if  the  variables  satisfy  a  semidefiniteness  constraint.  The  proof  exploits  the  relationship 
between  this  class  of  problems  and  quadratic  programs  with  complementarity  constraints. 

The  results  of  [A]  show  that  many  important  practical  problems  can  be  represented 
as  convex  optimization  problems  through  the  use  of  a  single  lifting.  This  includes  rank- 
constrained  semidefinite  programs.  The  latter  class  includes  problems  such  as  factor  anal¬ 
ysis  problems  for  finding  a  low-rank  covariance  matrix,  sensor  array  processing,  minimizing 
the  rank  of  a  Hankel  matrix  in  model  identification  problems  in  system  theory  and  signal 
processing,  problems  in  systems  and  control,  and  combinatorial  optimization  problems. 

The  convex  optimization  problem  is  not  easy  to  solve,  because  it  is  defined  over  a  convex 
cone  that  is  hard  to  work  with.  Nonetheless,  the  exact  reformulation  offers  the  possibility  of 
designing  new  and  effective  algorithms  for  solving  a  broad  class  of  optimization  problems. 

In  paper  [B],  we  developed  a  nonlinear  programming  formulation  of  the  problem  of  min¬ 
imizing  the  number  of  nonzero  components  in  the  solution  of  a  system  of  linear  equalities 
and  inequalities,  also  known  as  minimizing  the  LO-norm.  This  is  the  problem  of  compressed 
sensing  that  is  currently  of  great  interest.  It  also  arises  in  the  search  for  sparse  solutions 
to  support  vector  machine  problems  in  classification.  Our  method  is  slower  than  a  linear 
programming  approximation  to  the  LO-norm  problem,  but  it  has  the  advantage  that  it  often 
gives  a  sparser  solution  than  the  Ll-norm  approximation.  This  seems  to  be  especially  true 
when  the  system  involves  inequality  constraints,  as  is  the  case  for  sparse  support  vector 
machines. 


2  On  QPCCs,  QCQPs  and  Completely  Positive  Pro¬ 
grams 

The  material  in  this  section  is  drawn  from  our  paper  “On  QPCCs,  QCQPs  and  Com¬ 
pletely  Positive  Programs”  [2j.  The  quadratically  constrained  quadratic  program,  abbre¬ 
viated  QCQP,  is  a  constrained  optimization  problem  whose  objective  and  constraint  func¬ 
tions  are  all  quadratic.  The  specification  of  the  problem  also  allows  linear  constraints;  we 
also  allow  conic  convex  constraints  on  the  variables.  We  focus  on  QCQPs  that  may  have 
several  convex  quadratic  constraints  but  they  have  just  one  nonconvex  quadratic  constraint 
q(x)  <  0,  and  further  q(x)  >  0  for  any  x  that  satisfies  the  linear  and  conic  constraints: 
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minimize 

/0(x)  =  ( c°  )Tx  +  \  xTQ°x 

x£KnM 

subject  to 

q(x)  =  h  +  qrx  +  |xTQx  <  0 

(1) 

and 

fi(x)  =  ht  +  (c*)Tx  +  \xTQlx  <0,  i  — 

V"  ,1, 

where  M.  =  (x  G 

:  Ax  =  b},  where  x  G  /C  fl  M.  implies  q(x) 

>  0,  and  where  the 

constraints  fi(x)  <  0, 

i  —  1 . . . ,  I,  are  convex. 

We  denote  a  QCQP  of  this  form  an  nSp-QCQP,  because  it  has  no  Slater  point;  if  /  =  0 
then  we  call  it  an  nSpO-QCQP.  We  show  that  any  QCQP  can  be  expressed  in  this  form 
with  a  convex  objective  function,  generalizing  a  result  for  bounded  QCQPs  in  [9] .  Thus,  the 
considered  class  of  QCQPs  is  broad. 

Quadratic  programs  with  (linear)  complementarity  constraints  (QPCCs)  are  instances  of 
nSpO-QCQPs: 

minimize  cTx  +  |  xTQx 

X=(x°  jX1  ,x2) 


subject  to  Ax  =  b  and  (x1,  x2)  <  0 
with  x°  G  /C°,  x1  G  /C1,  x2  G  1C1* , 


(2) 


where  /C1*  is  the  dual  cone  to  1C1. 


Such  a  complementarity  constraint  renders  the  QPCC  a  nonconvex  disjunctive  program  even 
if  the  objective  function  is  convex. 

The  paper  addresses  several  topics  associated  with  the  QPCC  and  QCQP:  existence  of 
an  optimal  solution  to  a  QCQP,  the  formulation  of  a  QCQP  as  a  QPCC,  the  local  optimal¬ 
ity  conditions  of  a  class  of  quadratically  constrained  nonlinear  programs  failing  constraint 
qualifications,  and  the  formulation  of  a  QCQP  as  a  completely  positive  program.  The  paper 
contains  a  wealth  of  new  results  pertaining  to  the  three  classes  of  problems  appearing  in  the 
title;  see  Figure  [l]  These  results  are  all  for  the  situation  where  /C  is  a  convex  cone.  The 
upward  pointing  arrows  illustrate  relationships  where  the  lower  problem  can  be  regarded  di¬ 
rectly  as  an  instance  of  the  upper  problem.  The  downward  pointing  arrows  require  proof  and 
in  some  cases  the  cone  is  changed.  Collectively,  these  results  add  significant  insights  to  the 
problems.  Most  importantly,  our  study  touches  on  a  class  of  nonconvex  programs  failing  the 
Slater  constraint  qualification  and  suggests  that  such  problems  have  an  underlying  piecewise 
structure  and  can  be  converted  to  convex  programs  by  a  single  lifting  of  their  domain  of 
definition. 

The  central  problem  is  the  nSpO-QCQP;  some  QCQPs  are  already  in  this  form,  while 
any  other  can  be  manipulated  into  this  form.  The  manipulation  also  results  in  a  convex 
objective  function.  Provided  the  objective  is  copositive  on  a  certain  subset  of  the  recession 
cone  of  /C  D  M,  an  nSpO-QCQP  is  equivalent  to  a  convex  completely  positive  program. 
This  extends  the  recent  papers  m  that  address  the  completely  positive  representations  of 
binary  nonconvex  quadratic  programs,  certain  types  of  quadratically  constrained  quadratic 
programs,  and  a  number  of  other  NP-hard  problems.  Consequently,  we  show  that  any 
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QCQP 


Theorem  1 


T 


Binary  - >  QPCC  with  bounded  comple-  _  Convex  completely 

QP  4 -  mentarity  variables  [g]  positive  program 


Figure  1:  Diagram  of  results.  The  notation  “p  — >  qv  means  p  is  a  subclass  of  q.  Theorem, 
Proposition,  and  Corollary  numbers  refer  to  the  paper  [2j. 


QCQP  is  equivalent  to  a  convex  completely  positive  program ,  if  we  first  reformulate  it  as 
an  nSpO-QCQP  with  a  convex  objective  function.  Further,  a  rank  constrained  SDP  is  also 
equivalent  to  a  convex  completely  positive  program,  since  it  is  equivalent  to  a  QPCC  over  the 
semidefinite  cone,  with  a  linear  objective  function.  Similarly,  the  rank-sparsity  decomposition 
problem  is  equivalent  to  a  convex  program;  in  many  applications,  it  is  desired  to  split  a  matrix 
into  a  low  rank  part  and  a  sparse  part,  which  can  help  with  inference  m 

The  existence  of  an  optimal  solution  to  a  convex  QCQP  over  the  nonnegative  orthant  is 
fully  resolved  via  the  classical  Frank- Wolfe  theorem  [5] ,  which  states  that  such  a  minimization 
program,  if  feasible,  has  an  optimal  solution  if  and  only  if  the  objective  function  of  the 
program  is  bounded  below  on  the  feasible  set.  The  situation  with  a  nonconvex  QCQP  is 
rather  different;  the  sharpest  Frank- Wolfe  type  existence  results  for  a  feasible  QCQP  with  a 
nonconvex  (quadratic)  objective  are  obtained  in  [[25].  We  extend  their  work  as  follows: 

Theorem  1  Assume  the  cone  /C  is  polyhedral.  Then  the  FW  attainment  result  holds  for  the 
nSp-QCQP  (@)  if  either 

1.  I  <  1  or 
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2.  qo(x)  is  a  quasiconv ex  function  on  1C  C\M.. 


3  Complementarity  Formulations  of  4-norm  Optimiza¬ 
tion  Problems 


The  material  in  this  section  is  drawn  from  our  paper  “Complementarity  Formulations  of 
to-norm  Optimization  Problems”  [T8], 

Denoted  by  ||  •  ||o,  the  so-called  f'o-norm  of  a  vector  is  the  number  of  nonzero  components 
of  the  vector.  In  recent  years,  there  has  been  an  increased  interest  in  solving  optimization 
problems  that  minimize  or  restrict  the  number  of  nonzero  elements  of  the  solution  vector  [JJ 
m  eh  m  eh  da  m  es].  A  simple  example  of  such  a  problem  is  that  of  finding  a  solution 
to  a  system  of  linear  inequalities  with  the  least  ^o-norm: 


minimize  Iloilo 

ieR"  (3) 

subject  to  Ax  >  b  and  Cx  =  d, 

where  A  e  Mmxn,  C  G  Mfcxn,  b  e  Mm  and  d  €  Mfc  are  given  matrices  and  vectors,  respec¬ 
tively.  Since  this  problem  is  NP-hard,  one  popular  solution  approach  replaces  the  nonconvex 
discontinuous  Crnorm  in  ([3])  by  the  convex  continuous  id-norm,  leading  to  a  linear  program: 


minimize  II  a;  lb 

Igl" 

subject  to  Ax  >  b  and  Cx  =  d. 


(4) 


Theoretical  result  are  known  that  provide  sufficient  conditions  under  which  an  optimal  so¬ 
lution  to  Q  is  also  optimal  to  (|3|  [10-,  [IT]  19J  |33] .  Yet  these  results  are  of  limited  practical 
value  as  the  conditions  can  not  easily  be  verified  or  guaranteed  for  specific  realizations  of 
(J3|;  thus  in  general,  optimal  solutions  to  Q  provide  suboptimal  solutions  to  ([3]). 

It  is  our  contention  that,  from  a  practical  perspective,  improved  solutions  to  (j3])  can  be 
obtained  by  reformulating  the  Arllorm  in  terms  of  complementarity  constraints  m-  This 
leads  to  a  linear  program  with  linear  complementarity  constraints  (LPCC)  which  can  be 
solved  with  specialized  algorithms  that  do  not  depend  on  the  feasibility  and/or  boundedness 
of  the  constraints  [20],  2Tj .  In  the  event  that  bounds  are  known  on  the  solutions  of  the 
problem,  the  LPCC  can  be  further  reformulated  as  a  mixed- integer  linear  program  (MILP). 
However,  the  solution  of  this  MILP  is  usually  too  time-consuming  for  large  instances. 

As  an  alternative  to  the  MILP  approach,  the  LPCC  can  be  expressed  directly  as  a 
smooth  continuous  nonlinear  program  (NLP).  It  is  the  main  purpose  of  this  research  to 
examine  the  quality  of  solutions  computed  by  standard  NLP  solvers  applied  to  these  smooth 
reformulations  of  the  ^Q-norrn.  There  are  two  properties  of  the  NLP  reformulations  that 
make  them  difficult  to  solve.  First,  the  NLPs  are  highly  nonconvex,  and,  consequently, 
the  solutions  returned  by  the  NLP  solvers  depend  strongly  on  the  starting  point,  because 
the  NLP  methods  are  typically  only  able  to  find  local  minimizers  or  Karush-Kuhn-Tucker 
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(KKT)  points,  instead  of  global  minimizers.  Secondly,  the  NLPs  are  not  well-posed  in  the 
sense  that  they  do  not  satisfy  the  assumptions  that  are  made  usually  for  the  convergence 
analysis  of  standard  NLP  algorithms,  such  as  constraint  qualifications.  Nevertheless,  our 
numerical  results  show  that  these  methods  often  generate  high-quality  solutions  for  the  £q- 
norm  problem  (|3]). 

Together,  the  7'o-norm  and  its  complementarity  formulation  allow  a  host  of  minimization 
problems  involving  the  count  of  variables  to  be  cast  as  disjunctive  programs  with  comple¬ 
mentarity  constraints.  A  general  NLP  model  of  this  kind  is  as  follows:  for  two  finite  index 
sets  S  and  X, 

minimize  /(x)  +  y||  x  ||o 

X 

subject  to  Ci(x)  —  0,  i  G  S  (5) 

and  <y(x)  <  0,  !  6  I, 

where  7  >  0  is  a  prescribed  scalar  and  the  objective  function  /  and  the  constraint  func¬ 
tions  Ci  are  all  continuously  differentiable.  A  distinguished  feature  of  this  problem  is  that 
its  objective  function  is  discontinuous,  in  fact  lower  semi  continuous;  as  such,  it  attains  its 
minimum  over  any  compact  set,  but  in  general  the  existence/attainment  of  an  optimal  so¬ 
lution  is  not  immediately  clear.  Among  other  things,  the  reformulation  presented  below 
offers  a  constructive  venue  for  establishing  the  solvability  of  the  problem,  under  reasonable 
conditions. 

We  can  derive  an  equivalent  formulation  as  a  complementarity  problem: 


minimize  /(x)  +  7T  ( ln  —  £ ) 

X ,  X±,  £ 

subject  to  q(x )  =0,  i  e  8 
Ci(x)  <0,  i  6  I 


0  <  (  1  P  +  i  >0 
0  <  x+  1  x~  >  0 


and  £  <  ln, 


(6) 


where  we  have  used  an  arbitrary  given  positive  vector  7  instead  of  a  scalar  7-multiple 
of  the  vector  of  ones.  This  time,  the  statement  involves  nonlinear  objective  and  con¬ 
straint  functions,  giving  rise  to  a  Mathematical  Program  with  Complementarity  Constraints 
(MPCC).  It  is  not  difficult  to  deduce  that  if  x  is  an  optimal  solution  of  (J5|,  then  by  letting 
x±  =  max(0,  ±x)  and 


0  if  Xj  7^  0  | 

1  if  Xj  =  0  J 


j  =  I,-"  ,n, 


(7) 


the  resulting  triple  is  an  optimal  solution  of  (Jg])  with  objective  value  equal  to  ||x||o- 

Conversely,  if  is  an  optimal  solution  of  (J6|) ,  then  x  =  x+  —  x~  is  an  optimal  solution 
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of  ([5])  with  the  same  objective  value  as  the  optimal  objective  value  of  (|6]) .  The  definition  (J7|) 
provides  a  central  connection  between  (|5])  and  its  “pieces” . 

We  may  associate  with  (|5])  the  following  smooth  NLP  with  an  auxiliary  variable  £,  which 
we  call  a  half  complementarity  formulation: 

minimize  f(x)  +  7r  ( ln  —  £ ) 

x,£ 

subject  to  Ci(x)  =  0,  i  E  £ 

Ci(x )  <0,  i  G  X 
0  <  £  <  ln  and  £  °  x  =  0. 


The  paper  contains  theorems  relating  solutions  to  (J5])  with  KKT  points  for  (jb])  and  (J8]) . 
We  also  examine  relaxed  formulations  of  ©  and  ([8]) ,  and  show  interesting  convergence 
properties  as  the  relaxation  is  tightened.  In  particular,  we  can  show  for  certain  choices  of  / 
and  c  that  any  subsequence  of  local  minimizers  converges  to  a  solution  with  strong  theoretical 
properties. 

We  give  one  graph  to  give  a  flavor  of  the  computational  results  in  the  paper.  This  is  for 
a  problem  of  the  form 


minimize  ||x||o 

xei" 

subject  to  Ax  >  b  and  —  Mln  <  x  <  Mln, 


(9) 


where  A  e  Mmxn,  b  €  Mm,  and  M  >  0.  The  test  instances  were  generated  using  AMPL’s 
internal  random  number  generator,  where  the  elements  of  A  and  b  are  independent  uniform 
random  variables  between  -1  and  1.  Our  numerical  experiments  compare  the  performance  of 
different  NLP  optimization  codes  when  they  are  applied  to  the  different  NLP  reformulations. 
Because  these  problems  are  nonconvex,  we  also  explore  the  effect  of  different  starting  points. 

Our  numerical  study  suggests  that  standard  NLP  codes  are  able  to  generate  solutions  that 
can  be  significantly  better  than  those  obtained  by  convex  approximations  of  the  NP-hard 
problem.  This  is  somewhat  remarkable  because  the  NLP  formulations  are  highly  nonconvex 
and  the  usual  constraint  qualifications,  such  as  MFCQ,  do  not  hold.  Our  numerical  exper¬ 
iments  did  not  identify  a  clear  winner  among  the  different  reformulations  of  the  ^o-norm 
minimization  problems.  Similarly,  while  some  NLP  codes  tended  to  produce  better  results 
than  others,  it  is  not  clear  which  specific  features  of  the  algorithms  or  their  implementations 
are  responsible  for  finding  good  solutions.  We  point  out  that  each  software  implementation 
includes  enhancements,  such  as  tricks  to  handle  numerical  problems  due  to  round-off  error 
or  heuristics  that  are  often  not  included  in  the  mathematical  description  in  scientific  papers. 
Because  the  NLP  reformulations  of  the  f'o-problenrs  are  somewhat  ill-posed,  these  enhance¬ 
ment  are  likely  to  be  crucial  for  the  solver’s  performance.  Once  the  relevant  ingredients  of  the 
reformulation  and  optimization  method  have  been  identified,  it  might  be  possible  to  design 
specialized  NLP-based  algorithms  that  are  tailored  to  the  task  of  finding  sparse  solutions 
efficiently. 
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Figure  2:  The  sparsest  solutions  for  the  50  random  problems  using  different  NLP  solvers 
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Finally,  the  numerical  study  in  this  paper  has  been  performed  using  randomly  generated 
model  problems.  Future  efforts  will  explore  the  suitability  of  the  proposed  approach  for  £q- 
norm  optimization  problems  arising  in  particular  applications  areas  including  compressive 
sensing  mm,  basis  pursuit  HIE],  machine  learning  [131 126].  and  genome-wide  association 
studies  [32] . 

4  Algorithms  for  Global  Resolution  of  Mathematical 
Programs  with  Complementarity  Constraints 

4.1  General  methods  for  convex  MPCCs 

In  the  paper  “Obtaining  tighter  relaxations  of  mathematical  programs  with  complementarity 
constraints”  [[22],  we  investigated  cutting  plane  approaches  to  MPCCs.  In  an  effort  to  find  a 
global  optimum,  it  is  often  useful  to  examine  the  relaxation  obtained  by  omitting  the  com¬ 
plementarity  constraints.  We  discuss  various  methods  to  tighten  the  relaxation  by  exploiting 
complementarity,  with  the  aim  of  constructing  better  approximations  to  the  convex  hull  of 
the  set  of  feasible  solutions  to  the  MPCC,  and  hence  better  lower  bounds  on  the  optimal 
value  of  the  MPCC.  Better  lower  bounds  can  be  useful  in  branching  schemes  to  find  a  globally 
optimal  solution.  Different  types  of  linear  constraints  are  constructed,  including  cuts  based 
on  bounds  on  the  variables  and  various  types  of  disjunctive  cuts.  Novel  convex  quadratic 
constraints  are  introduced,  with  a  derivation  that  is  particularly  useful  when  the  number  of 
design  variables  is  not  too  large.  A  lifting  process  is  specialized  to  MPCCs.  Semidehnite 
programming  constraints  are  also  discussed.  All  these  constraints  are  typically  applicable 
to  any  convex  program  with  complementarity  constraints.  Computational  results  for  linear 
programs  with  complementarity  constraints  (LPCCs)  are  included,  comparing  the  benefit  of 
the  various  constraints  on  the  value  of  the  relaxation,  and  showing  that  the  constraints  can 
dramatically  speed  up  the  solution  of  the  LPCC. 

In  the  paper  “Using  quadratic  convex  reformulation  to  tighten  the  convex  relaxation  of 
a  quadratic  program  with  complementarity  constraints”  [3j,  we  looked  at  a  preprocessing 
technique  based  on  semidehnite  programming.  Quadratic  Convex  Reformulation  (QCR)  is  a 
technique  that  has  been  proposed  for  binary  and  mixed  integer  quadratic  programs.  In  this 
paper,  we  extend  the  QCR  method  to  convex  quadratic  programs  with  linear  complementar¬ 
ity  constraints  (QPCCs).  Due  to  the  complementarity  relationship  between  the  nonnegative 
variables  y  and  w,  a  term  yTDw  can  be  added  to  the  QPCC  objective  function,  where  D 
is  a  nonnegative  diagonal  matrix  chosen  to  maintain  the  convexity  of  the  objective  function 
and  the  global  resolution  of  the  QPCC.  Following  the  QCR  method,  the  products  of  linear 
equality  constraints  can  also  be  used  to  perturb  the  QPCC  objective  function,  with  the 
goal  that  the  new  QP  relaxation  provides  a  tighter  lower  bound.  By  solving  a  semidehnite 
program,  an  equivalent  QPCC  can  be  obtained  whose  QP  relaxation  is  as  tight  as  possible. 
In  addition,  we  extend  the  QCR  to  a  general  quadratically  constrained  quadratic  program 
(QCQP),  of  which  the  QPCC  is  a  special  example.  Computational  tests  on  QPCCs  are 
presented. 
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In  the  paper  “On  Convex  Quadratic  Programs  with  Linear  Complementarity  Constraints” 
[I],  we  looked  at  splitting  the  feasible  region  into  two  sets,  so  as  to  best  exploit  an  integer  pro¬ 
gramming  solver  and  a  logical  Benders  decomposition  approach.  The  paper  shows  that  the 
global  resolution  of  a  general  convex  quadratic  program  with  complementarity  constraints 
(QPCC),  possibly  infeasible  or  unbounded,  can  be  accomplished  in  finite  time.  The  method 
constructs  a  minrnax  mixed  integer  formulation  by  introducing  finitely  many  binary  vari¬ 
ables,  one  for  each  complementarity  constraint.  Based  on  the  primal-dual  relationship  of  a 
pair  of  convex  quadratic  programs  and  on  a  logical  Benders  scheme,  an  extreme  ray/point 
generation  procedure  is  developed,  which  relies  on  valid  satisfiability  constraints  for  the  in¬ 
teger  program.  To  improve  this  scheme,  we  propose  a  two-stage  approach  wherein  the  first 
stage  solves  the  mixed  integer  quadratic  program  with  pre-set  upper  bounds  on  the  comple¬ 
mentarity  variables,  and  the  second  stage  solves  the  program  outside  this  bounded  region  by 
the  Benders  scheme.  We  report  computational  results  with  our  method.  We  also  investigate 
the  addition  of  a  penalty  term  yDw  to  the  objective  function,  where  y  and  w  are  the  com¬ 
plementary  variables  and  D  is  a  nonnegative  diagonal  matrix.  The  matrix  D  can  be  chosen 
effectively  by  solving  a  semidehnite  program,  ensuring  that  the  objective  function  remains 
convex.  The  addition  of  the  penalty  term  can  often  reduce  the  overall  runtime  by  at  least 
50%.  We  report  preliminary  computational  testing  on  a  QP  relaxation  method  which  can  be 
used  to  obtain  better  lower  bounds  from  infeasible  points;  this  method  could  be  incorporated 
into  a  branching  scheme.  By  combining  the  penalty  method  and  the  QP  relaxation  method, 
more  than  90%  of  the  gap  can  be  closed  for  some  QPCC  problems. 


4.2  Bi-parametric  linear  complementarity  constrained  linear  pro¬ 
grams 


In  the  paper  “An  Algorithm  for  Global  Solution  to  Bi-Parametric  Linear  Complementarity 
Constrained  Linear  Programs”  [22]  we  consider  the  following  problem.  Given  variables  x  G 
Mn  and  y  G  Mm  and  parameters  q  G  Mm,  N  G  MmXTl  and  M  G  Mmxm,  the  complementarity 
constraints 


0  <  y  T  w  :=  q  +  Nx  +  My 


can  be  represented  equivalently  using  the  nonconvex  quadratic  constraint 

qTy  +  yT My  +  yT  Nx  <  0 


together  with  nonnegativity  restrictions  on  y  and  w.  In  this  paper,  we  focused  on  problems 
with  n  =  2  and  with  M  positive  semidehnite,  a  situation  that  arises  often  in  practice  (see  [[23] 
for  example).  We  developed  a  domain  partitioning  scheme  for  the  problem,  branching  on  x , 
and  proved  finite  convergence.  We  obtained  promising  computational  results  on  our  test 
problems. 


4.3  Parameter  selection  in  support  vector  machine  regression 

The  support  vector  machine  (SVM)  regression  problem  has  two  design  parameters:  C  to 
control  the  regularization  and  e  to  control  the  width  of  a  regression  tube.  Previously,  we 


11 


have  developed  an  LPCC  formulation  of  the  problem  of  choosing  the  two  parameters  [5] . 
In  the  paper  “Global  Resolution  of  the  Support  Vector  Machine  Regression  Parameters 
Selection  Problem”  [23],  we  apply  the  algorithm  of  [22]  to  solve  the  parameter  selection 
problem.  Through  the  use  of  the  algorithm,  we  are  able  to  find  globally  optimal  values  of  C 
and  £. 


5  Other  Results 

In  the  paper  “Convex  Quadratic  Relaxations  of  Nonconvex  Quadratically  Constrained  Quadratic 
Programs”  [SHj;  we  extend  the  technique  introduced  in  |2HJ  for  MPCCs  to  nonconvex  QCQPs. 
Nonconvex  quadratic  constraints  can  be  linearized  to  obtain  relaxations  in  a  well-understood 
manner.  We  propose  to  tighten  the  relaxation  by  using  second  order  cone  constraints,  re¬ 
sulting  in  a  convex  quadratic  relaxation.  Our  quadratic  approximation  to  the  bilinear  term 
is  compared  to  the  linear  McCormick  bounds.  The  second  order  cone  constraints  are  based 
on  linear  combinations  of  pairs  of  variables.  With  good  bounds  on  these  linear  combina¬ 
tions,  the  resulting  constraints  strengthen  the  McCormick  bounds.  Computational  results 
are  given,  which  indicate  that  the  convex  quadratic  relaxation  can  dramatically  improve  the 
solution  times  for  some  problems. 

We  develop  a  homotopy  approach  to  finding  the  solution  to  an  LPCC  in  the  paper  “A 
Globally  Convergent  Probability-One  Homotopy  for  Linear  Programs  with  Linear  Comple¬ 
mentarity  Constraints”  [ST].  A  solution  of  the  standard  formulation  of  a  linear  program 
with  linear  complementarity  constraints  (LPCC)  does  not  satisfy  a  constraint  qualification. 

A  family  of  relaxations  of  an  LPCC,  associated  with  a  probability-one  homotopy  map,  pro¬ 
posed  here  is  shown  to  have  several  desirable  properties.  The  homotopy  map  is  nonlinear, 
replacing  all  the  constraints  with  nonlinear  relaxations  of  NCP  functions.  Under  mild  ex¬ 
istence  and  rank  assumptions,  (1)  the  LPCC  relaxations  RLPCC(A)  have  a  solution  for 
0  <  A  <  1,  (2)  RLPCC(l)  is  equivalent  to  LPCC,  (3)  the  Kuhn- Tucker  constraint  qualifica¬ 
tion  is  satisfied  at  every  local  or  global  solution  of  RLPCC(A)  for  almost  all  0  <  A  <  1,  (4) 
a  point  is  a  local  solution  of  RLPCC(l)  (and  LPCC)  if  and  only  if  it  is  a  Kuhn- Tucker  point 
for  RLPCC(l),  and  (5)  a  homotopy  algorithm  can  find  a  Kuhn- Tucker  point  for  RLPCC(l). 
Since  the  homotopy  map  is  a  globally  convergent  probability-one  homotopy,  robust  and 
efficient  numerical  algorithms  exist  to  find  solutions  of  RLPCC(l).  Numerical  results  are 
included  for  some  small  problems. 

In  the  paper  “Rebalancing  an  Investment  Portfolio  in  the  Presence  of  Convex  Transaction 
Costs  including  Market  Impact  Costs”  [27],  we  look  at  a  class  of  QCQPs  arising  in  financial 
optimization.  The  inclusion  of  transaction  costs  is  an  essential  element  of  any  realistic 
portfolio  optimization.  In  this  paper,  we  extend  the  standard  portfolio  problem  to  consider 
convex  transaction  costs  that  are  incurred  to  rebalance  an  investment  portfolio.  Market 
impact  costs  measure  the  effect  on  the  price  of  a  security  that  result  from  an  effort  to  buy 
or  sell  the  security,  and  they  can  constitute  a  large  part  of  the  total  transaction  costs.  The 
loss  to  a  portfolio  from  market  impact  costs  is  typically  modeled  with  a  convex  function  that 
can  usually  be  expressed  using  second  order  cone  constraints.  The  Markowitz  framework  of 
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mean- variance  efficiency  is  used.  In  order  to  properly  represent  the  variance  of  the  resulting 
portfolio,  we  suggest  rescaling  by  the  funds  available  after  paying  the  transaction  costs.  This 
results  in  a  fractional  programming  problem,  which  can  be  reformulated  as  an  equivalent 
convex  program  of  size  comparable  to  the  model  without  transaction  costs.  An  optimal 
solution  to  the  convex  program  can  always  be  found  that  does  not  discard  assets.  The 
results  of  the  paper  extend  the  classical  Markowitz  model  to  the  case  of  convex  transaction 
costs  in  a  natural  manner  with  limited  computational  cost. 
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